Search AI Products and News

The ByteDance Doubao large model team announced today the successful development of a new sparse model architecture called UltraMem. This architecture effectively addresses the high memory access issues during the inference of MoE (Mixture of Experts) models, improving inference speed by 2 to 6 times compared to MoE, and reducing inference costs by up to 83%. This groundbreaking advancement opens a new path for efficient inference of large models. The UltraMem architecture successfully resolves the memory bottleneck during inference of MoE architectures while maintaining model performance. Experimental results show that the parameters and activation conditions are the same.

At the 2024 Tencent Global Digital Ecosystem Conference, Tencent's Vice President and Head of Cloud Business, Qiu Yuepeng, announced the official launch of the Tencent Hunyuan Turbo large model. This new generation of large models is designed based on the MoE (Mixture of Experts) architecture, achieving a doubling in inference efficiency with a 100% improvement compared to the previous generation, while significantly reducing inference costs by 50%. Additionally, Hunyuan Turbo also demonstrates outstanding performance in decoding efficiency, improving by 2.

The daily call volume of the Wenxin large model has exceeded 50 million, with a quarterly growth rate of 190%. Approximately 26,000 enterprises are using the Wenxin large model, showing a quarterly growth rate of 150%. Well-known companies such as Samsung, Honor, and Autohome have collaborated with Baidu. Baidu continues to reduce the inference costs of the Wenxin large model, which have now dropped to 1% of the level from March last year. An increasing number of users and enterprises are starting to use the Wenxin large model.

The inference cost of Baidu Wenxin large model has been reduced to 1% of the original. The daily call volume of the Wenxin large model exceeds 50 million, with a quarterly growth rate of 190%. Approximately 26,000 enterprises are using the Wenxin large model, with a quarterly growth rate of 150%.